PROCESS FOR RECOGNITION OF TRACK MARKERS USING IMAGE DATA 

\ 

The invention concerns a process according to the pre- 
characterizing portrpn of Patent Claim 1. 

Camera based systems for automatic vehicle tracking and for 
warning of unintentional deviations from the vehicle track or 
lane depend upon a reliable recognition of the vehicle lane 
boundaries. Typically, these boundaries are marked by means of 
10 continuous or interrupted lines. It is also known (for example: 
in the USA) to mark vehicle lane boundaries by a sequence of 
points (terrain points) . 

In US Patent 5,351,044, a process for recognition of vehicle lane 
y> boundaries is disclosed, which evaluates pixel-based image data. 

TSX? 

u -=j Therein, those image points of which the intensity or brightness 

^ significantly exceeds the intensity or brightness of the 

!,l surrounding or ambient points are assigned to a border. 

20 Further, a process is known which evaluates the correlation of 
the bordering image cells within the image data for detection of 
it the vehicle lane boundaries. For this it is however necessary to 
transform the perspective camera data into non-perspective image 
data . 
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US Patents 5,517,412 and 4,868,752 show processes for detection 
of the vehicle lane boundaries, which are based on the 
segmentation of linear elements by means of line detectors ("edge 
detection") with subsequent Hough -trans format ion for 
30 reconstruction of the boundary. Similar technology is disclosed 
in US Patent 4,970,653, wherein here the segmentation of the line 
elements is supported by supplemental template matching. 
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In the process proposed in US Patent 6,091,833, the area in which 
linear detection is carried out is curtailed by the selection of 
areas which, transformed in the frequency space, exhibit a high 
intensity by their low frequency components. 
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"SThe task of thk invention is to provide a new type of process 
f according to the p^re- characterizing portion of Patent Claim 1. 

The task is solved \by a process, which is described by the 
10 characteristic features of Patent Claim 1. Advantageous 
embodiments and further developments of the invention are set 
forth in the dependent \claims and described on the basis of 
illustrative embodiments and figures. 

335 Figure 1 shows an image section 10 with point-shaped vehicle lane 

y3 

%a markings 11 and selected areas (ROD 12 • 
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yj Figure 2 shows a flow diagram with respect to the recognition of 

irfz 

~ the point-shaped vehicle lane markings 11. 
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Figure 3 shows a flow diagram concerning the binarization or 
digitization of the image point (pixel) which is associated with 
the vehicle lane boundary 11. 



25 Figure 4 shows an example of a gray- scale or tonal value 
processing of image data, wherein the regions evaluated by the 
matched- filter are extracted. 



In the inventive process, the morphological characteristics of 
30 point -shaped vehicle lane markings are considered or evaluated on 
the basis of a priori or common knowledge. Therein, beginning 

with a starting image, a multi-step search is made for point - 
shaped vehicle lane markings ("Bot Dot Markings"). 
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In Figure 1 there is shown a preferred manner, in which for 
minimizing the processing complexity, and, at the same time, for 
reduction the false alarm likelihood, areas (ROI) 12 to be 

5 processed are separated out from the totality of the image data 
10. This occurs beginning with a priori knowledge with the goal, 

to select areas 12, which with high probability contain vehicle 

lane markings, so called ROI (regions of interest) . Herein the 
positioning of the ROI occurs in effective manner on the basis of 

10 the known camera geometry (height, tilt angle,. . . ) as well as 
on the basis of other perimeters such as for example the geometry 
of the vehicle track, the dimensions of the vehicle lane markings 
or the vehicle position within the vehicle lane. Herein it is 

Q generally assumed, that the vehicle is situated between left and 

JT5 right markings, and the vehicle lane breadth meets certain 

□ standards . 

Ly 

jjf} In Figure 2 there is shown, as an example, the individual steps 

£n of the inventive image data processing beginning from camera 

20 image 10, which still contains the entire image information, up 

tfi& to the extraction of the coordinates for individual elements of 

P 

U the vehicle lane marking, which in general correspond to the 

Q coordinates of marking points. 

□ 

25 In a first step 21 areas (ROI) 12 are separated out from the 

entirety of the image data 10 to be processed. For this, in the 

initialization of the ROI 12 in the framework of the inventive 

process model parameters are varied so long until vehicle lane 
markings are found. Therein, as model parameters there are taken 
30 into consideration for example information such as the breadth of 
the vehicle lane, the positioning or orientation of the camera 
with respect to the center of the vehicle lane or the yaw angle 
of the vehicle. In the later course of the process, for 
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repositioning of already initialized ROI 12, parameter 

predictions from a vehicle -street -model are drawn upon on the 
principle of a prediction of an evaluation process for parameter 
determination. For this, evaluation processes are advantageous 
5 which are based upon a Kalman-f ilter . In the repositioning of 
the ROI 12, it is in advantageous manner controlled by the 

variation of the result value of the prediction of the evaluation 
process for parameter determination, wherein the width is adapted 
proportionally to the size of the variance of the results. It is 
10 likewise advantageous to limit the ROI 12 vertically on the basis 

of a minimal and a maximal distance in the street plane. This is 
particularly useful when the inventive process is employed at 
night. Herein in advantageous manner the vertical limitation of 
f » the ROI 12 is determined by the range of the maximal illumination 

"ft (high beam, low beam) . It is conceivable that the regulation is 

~i controlled by the number of the image points expected to be 

=p associated with a vehicle lane marking, wherein this regulation 

'tt is then considered as optimal when the number of the image or 

£fl measuring points to be expected is maintained constant over all 

20 distance ranges. 

j.+ 

□ According to the number of suitable ROI 12, a matched- filter is 
H employed in process step 22, in order to better extract the image 
w points (pixels) associated with a vehicle lane marking from the 
25 background. Herein the matched- filter is advantageously adapted 

to the shape and size of the vehicle lane marking and/or the 

statistics of the background being searched. 

In particularly advantageous manner, the matched filter is so 
30 arranged that in the framework of its application in the 
environment of the position being examined, the average gray 
scale or tone value (hereafter gray value) of the background is 
measured, and that on the presentation of an image point, which 
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is potentially to be associated with a vehicle lane marking, is 
confined on the basis of a comparison between background noise, 
the average gray value in the environment, and the gray value of 
the position to be searched. In general, for this the filter is 
5 implemented in separated geometry or model -type, in which the x-y 
components are presented separately. It has however been found, 
that for location or determination of point -shaped vehicle lane 
markings in the most cases, the processing complexity can be 
reduced, when in the evaluation of the matched filter only the x- 

10 components are considered. For this, the average value and the 
standard deviation right and left of a position to be examined 
are measured. If the gray value exceeds the average value of the 
intensity of the background at more than the comparative 
threshold determined from the white noise of the background and 

1:5 threshold determination, the position is marked as potentially 
belonging to a vehicle lane marking. 

jsssi 
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hi Figure 4 shows, as an example, the gray value curve 40 over an 

fa 

!^ image cell to be evaluated within the image data or, as the case 

20 may be, the ROI, wherein in this example only the x components 

H 3 are taken into consideration. For evaluation of the image point 

?~ 41 on the position Xq of the image cell, the intensity thereof as 

? ~ well as the statistical information of two same size areas SI and 
?*\ S2, which are situated at a distance d of xO therefrom, are 
25 evaluated. The distance d is preferably selected to be a value, 
which is predicated or predicted by the evaluation process for 
parameter determination in process step 21 (Fig. 2) as the to be 

expected width of an element of the vehicle lane marking. It is, 
on the other hand, however also possible to assign the distance d 
30 in the framework of the inventive process a fixed value (for 
example an expected value) and to so leave it. The size of the 
area or range or field SI and S2 is in advantageous manner 
selected to be identical, wherein their measurements, if 
possible, should not be larger than the predicated or predicted 
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value of an element of the vehicle lane marking. If this 
predicted value is however so small that no useful statistical 
evaluation of the values contained in SI or, as the case may be, 
S2 any longer appears possible, then the areas or ranges SI or, 
5 as the case may be, S2 can also be selected to be larger. In 
practice, in the predication of the width of the elements of the 
vehicle lane markings of two pixels, the selection of five pixels 
as the width of areas SI and S2 has been found to be effective. 



10 From the values within areas or ranges SI or S2, the average 
values Al or, as the case may be, A2 and the standard deviation 
al or, as the case may be, a2 is calculated. The standard 
deviation al or, as the case may be, g2 is subsequently employed 

for calculation of the threshold values Tl or, as the case may 
C3 ... 

IS be, T2 . Therein, it is conceivable, m a first step or statement 

N or set-up to select the threshold values according to the three - 

tl 

1» fold of the respective standard deviations, thus to select Tl = 3 

W * al and T2 = 3 * g2 . In particularly advantageous manner, the 

lf\ thus generated threshold values are corrected by means of a 

20 threshold value controller or regulator 25 described in greater 

5 — ' 

detail in the following. After the determination of the 
H threshold values Tl or, as the case may be, T2 and their 
U subsequent correction, the support points for the comparison 
P threshold 42 is calculated by addition of Tl and Al or, as the 

25 case may be, T2 and A2 . Herein the center of the area SI and S2 
is advantageously selected as the x-position of the support 
point. Subsequently, the value 41 at the position Xq can now be 

compared with the value of the comparison threshold 42 and the 
comparison result can in inventive manner be drawn upon for 
30 identification of elements of the vehicle lane marking. 

The image point (pixel) identified and extracted by the matched- 
filter, which is associated with vehicle lane markings, are 
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digitized (conversion to binary) in processing steps 23, 24, and 

25 and collected into groups. In the digitization, the intensity 

of the individual pixels are compared with a threshold value and 
then the compared pixels are only drawn upon for a further 
5 processing when their intensity exceeds this threshold value. In 
Figure 3, the inventive process step for digitization is 
elucidated on the basis of a flow diagram. Herein the threshold 
value Tl and T2 is determined from the background noise in 
advantageous manner by means of a threshold value controller 25. 

10 For the determination thereof, it is useful when the threshold 
value controller draws upon a priori knowledge about the expected 

surface 35 occupied by the vehicle lane marking, which correlates 

directly with the expected number of the vehicle lane marking 
□ associated image points (pixels) . In this manner, the threshold 
jft value controller 25 can be so set up or arranged, that it aims in 

O directed manner to relate the number in the ROI 12 extracted 

j,j image points as precisely as possible to the expected value 
CR thereof. After the extraction of image points potentially 

fn 

_ belonging to a vehicle lane marking and their subsequent 
E0 digitization 33, these are collected in process step 24 for a 

f~ further processing (for example within 35) for marking objects 

(pixel groups) . 

Q 

After the generation of marking objects, these can be examined in 
25 process step 27 with respect to their correspondence with the 

known morphological characteristics of the vehicle lane markings. 
Herein in advantageous manner for example the size of the marking 
objects, their roundness, the individual image points (pixels) , 
or the number of free spaces within the marking objects 
30 (compactness, clustering) are evaluated, as to whether they can 
satisfy the criteria of the morphological characteristics of the 
vehicle lane markings on the basis of a priori knowledge. Based 



- 7 - 



» # 

upon this determination, each pixel group, which satisfies the 
criteria of a vehicle lane marking, is considered to be an actual 
marking object and is characterized by the image coordinates. 
For this, in advantageous manner, the coordinates of the center 
5 of gravity of the marking image associated pixel group can be 
selected as the characterizing image coordinates. 

After successful characterization of the marking object, the 
image coordinates can be employed, using curve regression, in 

10 order to describe the boundaries of the own vehicle track or in 
certain cases, the course of the vehicle track as well as the own 
position with respect to the the vehicle lane center. It is then 
particularly useful to supply this description to the inventive 
evaluation process for parameter determination (for example a 

935 Kalman- filter) for repositioning of the ROI within the image 

y3 data. 

The invention process is, of course, not limited to the 
p recognition of dot-shaped vehicle lane markers. It is also 
2® possible therewith, for example, to recognize continuous linear 
r. vehicle lane markings even when these have been strongly degraded 

\>* by changes or wear. 

□ 
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